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ABSTRACT 

We study a collection of heterogeneous XML databases maintain- 
ing similar and related information, exchanging data via a peer to 
peer overlay network. In this setting, a mediated global schema 
is unrealistic. Yet, users/applications wish to query the databases 
via one peer using its schema. We have recently developed Hep- 
ToX, a P2P Heterogeneous XML database system. A key idea is 
that whenever a peer enters the system, it establishes an acquain- 
tance with a small number of peer databases, possibly with dif- 
ferent schema. The peer administrator provides correspondences 
between the local schema and the acquaintance schema using an 
informal and intuitive notation of arrows and boxes. We develop a 
novel algorithm that infers a set of precise mapping rules between 
the schemas from these visual annotations. We pin down a seman- 
tics of query translation given such mapping rules, and present a 
novel query translation algorithm for a simple but expressive frag- 
ment of XQuery, that employs the mapping rules in either direction. 
We show the translation algorithm is correct. Finally, we demon- 
strate the utility and scalability of our ideas and algorithms with a 
detailed set of experiments on top of the Emulab, a large scale P2P 
network emulation testbed. 

Categories and Subject Descriptors 

H. 2.4 [Systems]: Distributed Databases, Query Processing; H.2.5 
[Heterogeneous Databases]: Data Translation 

Keywords 

XML Mappings, XML Query Translation, Heterogeneous Peer-to- 
peer XML Data Management 

I. INTRODUCTION 

Consider a large scale data sharing setting where several XML data 
sources in a similar domain (e.g., hospitals/medical centers) need 
to share (parts of) their data. The source schemas can be quite dif- 
ferent. Figure 1 shows an example of two hospital DTDs expressed 

*This paper was submitted to VLDB 2005. An earlier version was 
submitted to SIGMOD 2005. 



as graphs, adapted from [18]. (See [3, 17] for similar healthcare ex- 
amples of P2P DBMS.) Ignore the cross arrows linking the DTDs 
for now. The first DTD shows how information is represented in 
the Montreal General Hospital database. Every patient 
is assigned a unique ID by the hospital and has a unique Medi- 
care number (MedCr#). Note that symptoms of problems expe- 
rienced by patients and the treatments they are administered are 
all grouped under patients. Data pertaining to patient admissions 
is maintained separately and is captured via the ID/IDREF link 
@PatRef^@ID (shown as a solid grey arrow in Figure 1). 

On the other hand, a patient who is moved from Montreal to Boston 
needs to establish early enough an acquaintance with the Mass 
General Hospital schema. Indeed, the patient database at the 
Mass General Hospital in Boston is organized quite differ- 
ently. In this schema, at admission time patients are classified on 
the basis of their main complaint (pulmonary, coronary, or other 
illnesses not shown in the figure). Under this classification, the 
usual patient details such as name, hospital ID, insurance number, 
insurer, admission and discharge dates are stored. Progress of pa- 
tients during their stay in the hospital is recorded: patients' his- 
tory of health problems as well as the treatments administered are 
tracked. All of this information is connected to the patients via the 
ID/IDREF link SPatRef — > @ ID (solid grey arrow, Figure 1). 

Suppose we have several heterogeneous schemas, which describe 
the data of several health/medical data sources. A natural strat- 
egy for data exchange between the sources is to embed them in a 
loosely coupled environment. The loose coupling is crucial for al- 
lowing maximum flexibility, with no hard commitments to conform 
to any rigid requirements. Currently, there is significant interest in 
peer-to-peer (P2P) database systems [2, 3, 5, 14, 18, 20, 24]. A P2P 
database system is essentially a collection of autonomous database 
systems connected by a P2P network, with the flexibility that peers 
may enter or leave the network at will. In this paper, we assume 
that any peer at the time of entering the P2P network provides a 
mapping between its schema and a small number of the existing 
peer schemas. The peers chosen by the entering peer are called 
its acquaintances [2]. E.g., the Montreal General peer may 
provide "correspondences" between the concepts (elements and at- 
tributes) in its schema and those in the Mass General schema. 
The next question is who will provide this mapping and how. We 
could make use of (semi-)automatic schema mapping tools [9, 21, 
13]. However, one might wish to customize or even override the 
associations produced by these tools. To allow for this, we con- 
sider a model where a peer database administrator supplies simple 
and intuitive correspondences between the peer schema and a small 
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Figure 1: Mapping two Heterogeneous Peer DTDs. To minimize clutter, some arrows between corresponding tags have been omitted 
(e.g., @ID to @ID); every unlabeled edge is labeled '1' by default. 



number of acquaintance schemas. The correspondences are spec- 
ified in the form of arrows and boxes, illustrated in Figure 1, and 
will be explained in Section 2. We use the correspondences as a 
basis for automatically inferring a mapping expression, expressed 
in the form of Datalog-like rules. The peer DBA may (but doesn't 
have to) examine the rules and make any necessary adjustments. In 
this way, joining a P2P DBMS is made a lightweight operation. 

There may be considerable differences in the way the peers orga- 
nize their data, including differences in data representations (e.g., 
stock names instead of ticker symbols, different units, etc.), as well 
as differences in underlying schemas (e.g., group treatments un- 
der the patients receiving them as opposed to maintaining separate 
lists of patients and treatments and linking them with ID/IDREF 
links). Differences in data representation were addressed by previ- 
ous work [14] by introducing the so-called mapping tables. These 
are aimed at mapping value aliases in P2P networks, which are or- 
thogonal to schema mappings. Differences in schemas have also 
been studied in Clio [20] and in Piazza [3, 4], However, [20] 
focuses on deriving mappings across schemas and uses them to 
translate the data accordingly. Query translation across the inferred 
mappings is not considered. Furthermore, the kind of heterogeneity 
addressed in [20] is limited. Piazza [3, 4] addresses the schema me- 
diation problem within a community of peers, where entering peers 
establish semantic mappings with existing peers, in an XQuery-like 
language. Our mappings are to be considered in a large P2P frame- 
work. For this reason, we do not expect a peer DBA/user to know 
the rule machinery at all, whereas it is reasonable to assume s/he 
can draw arrows and boxes. Piazza is again limited in the extent 
of schema heterogeneity it can handle. A detailed comparison with 
this and other related work appears in Section 6. 

Our further goal is to permit users and applications of any peer 
database to access data items of interest by simply posing a query to 
their peer, regardless of the location of the data items or the schema 
under which they are organized. In other words, the existence of 
numerous peers and their schemas should all be transparent to the 
user/application posing the query. E.g., for answering the query 
"what are the treatments administered to patients admitted with a 
coronary illness?", we want to manipulate data from all peers "vis- 
ible" to the original peer, containing logically relevant information. 
Clearly, queries posed to a peer need to be translated appropriately 
so as to run on other peers. 

The major challenges we address are the following: (1) How do we 
cope with the heterogeneity in the peer schemas in the context of 



P2P data sharing? (2) What is the exact semantics for the evalua- 
tion of peer queries, and how do we translate queries across peers, 
given the heterogeneity of their schemas? (3) How can we evaluate 
queries correctly and efficiently over the network? 

Summarizing, we make the following contributions. 

• We assume correspondences between a pair of peer DTDs 
are specified diagrammatically. We develop a novel algo- 
rithm for inferring mappings between the DTDs, couched in 
the form of Datalog-like rules and discuss the significance of 
the class of transformations captured by the rules (Section 3). 

• We define the semantics of peer queries. We develop a novel 
query translation algorithm that handles a simple but signifi- 
cant fragment of XQuery and show that it is correct w.r.t. the 
above semantics. We illustrate our algorithm with examples 
(Section 4). Translation is non-trivial even for the XQuery 
fragment considered. 

• We have developed HepToX, 1 a HEterogeneous Peer TO 
peer Xml database system, incorporating the ideas developed 
in this paper. We describe our implementation of HepToX, 
including the strategies employed for efficient query evalua- 
tion. We ran an extensive set of experiments to measure the 
effectiveness of our query translation algorithm, as well as 
the scalability of our approach. We discuss the results and 
the lessons learned (Section 5). 

Section 2 gives a motivating example. Related work is discussed 
in Section 6, while Section 7 summarizes the paper and discusses 
future research. 

2. A MOTIVATING EXAMPLE 

Revisit the example discussed in the introduction (Figure 1). The 
arrows provide informal correspondences between the two hospital 
DTDs, and are illustrated with different types of arrows, for clarity. 
Henceforth we refer to the Montreal General DTD as MonG and 
the Mass General DTD as MasG. The arrows capture simple 1-1 
correspondences between terms such as "MedCr# in the first DTD 
to Policy# in the second" and "Name in the first to Patient 
in the second". The correspondences can be naturally understood 
as propagating from the leaves up to the their parent/ancestor el- 
ements as in [9]. Arrows between leaf elements of tree DTDs 

'Pronounced Hep Talk: heterogeneous peers talk! 



1. MassGeneral — > fl($Mgh) [Admission -t f2($Mgh) 

[SAP/textQ -k /3($AP/4ext(), $/D, $M, $AD, SDD, $N) 
[QID -> $/D, Policy# -> SM, 

Enter — » SAD, Leave -> SDD, Patient — > $N]]] 



MonGenHosp s $Mgh[Patient — > $P 

[@ID -> SID, MedCr# -> SM, Name -> $7V] , 
Admission — ► $A[Problem -> SAP, AdmDate -» SAD, 
DisDate -> SDD, ©PatRef -> $PR]], $PR = SID. 

2. MassGeneral -> fl{$Mgh) [Progress -> /2($PP) 

[Symptom -> /3($DP, $DD)[Date -> $ED, Desc -> SEP]]] 



MonGenHosp — > SMgh [Patient — > SP 

[@ID -> $/D, MedCr# -> $Af , Name -> $7V, Hist -> $ff 
[Event -> SE [Problem -> SEP, Date -» $ED}]] 
Admissions $A[Problems SAP, AdmDate — > SAD, 
DisDate -> SDD, (BPatRef -» SPR]], SPR = $ID. 

3. MassGeneral -> /l($A/g/t)[Progress -> /2($PP) 
[Treatment -> /3($TDate, STDesc) 

[Date -> STDate, Desc -> STDesc]]] 



MonGenHosp — * $Mgh[Patient — > $P 

[@ID -> $PD, MedCr# -> $M, Name -> $N, 

Treat -> $T[Date -> STDate, Desc -> STDesc]], 
Admissions $A[Problems SAP, AdmDate — > SAD, 
DisDate -> SDD, (BPatRef -» $PPJ], $PP = $ID. 

4. MassGeneral -► /l(JMjk) [Progress -» f2($ID) [QPatRef -> $PP,]] 



MonGenHosp — > $Mg/i[Admission — > $ A [Problem — > SAP, 

AdmDate -» SAD, DisDate -> SDD, OPatRef -> $PP], $PP = SID 



have a straightforward meaning. Since our DTDs are DAGs, we 
use different arrow types for disambiguation. E.g., consider the 
correspondence between Desc in the two DTDs. Desc has a 
unique parent in first DTD while it has two parents in the sec- 
ond. For disambiguation, we use the same arrow type for the edges 
Treat/Desc in MonG, Treatment/Desc in MasG, and the ar- 
row connecting them. Similarly, we use the same arrow type for 
the edges Event/Problem in MonG, Symptom/Desc in MasG, 
and the arrow connecting them. Other arrows can be understood 
similarly. E.g., Event/Date and Treat/Date are matched to 
their counterparts in the second DTD. Finally, consider the (thick 
dashed) box enclosing the Pulmonary and Coronary nodes. 
There is a thick dashed arrow matching Admission/Problem 
to this box. What it says is that 'Pulmonary' and 'Coronary' cor- 
respond to values of the element Admission/Problem in the 
first database. Again, recall there may be other illnesses (corre- 
sponding to values of Admission/Problem) not shown in the 
figure for brevity. In effect, illnesses which may be instances of 
Admission/Problem in the first database correspond to tags in 
the second database, but there is no assumption that the set of ill- 
nesses occurring in the two databases are the same or even overlap. 

Boxes are used to group together tags of nodes in a DTD that cor- 
respond to instances of a tag in another DTD, as part of the corre- 
spondence specification. Arrows specify a simple one-to-one cor- 
respondence between the identified concepts. However, arrows and 
boxes in and of themselves do not teJJ us how a database that con- 
forms to a DTD may be transformed to one that conforms to the 
other DTD. Why do we care about this transformation any way? 
The reason is that this transformation is closely tied to the seman- 
tics of query answering, as we will see in Section 4. 1 . While we 
will give an algorithm for inferring mapping expressions from cor- 
respondences specified using arrows and boxes, here we informally 
explain the mapping between the two DTDs in Figure 1. 

Note that in the MonG DTD, patients' history (problems and dates) 
and the treatments they undergo are both nested under patients. All 
admission information is maintained separately and linked to the 
appropriate patient via the patient's ID. In the MasG database, treat- 
ment and history information (symptoms) is separated out from pa- 
tients and linked to them via their ID. Additionally, patients are 
represented along with the rest of the admission data, but this data 
is classified based on the type of problem/illness identified at the 
time of admission. Whatever mapping expressions we use should 
be capable of transforming instances of one of these schemas/DTDs 
to those of the other. 

Our algorithm, to be given in Section 3, will produce the map- 
ping expression shown in Figure 2, expressed as Datalog-like rules, 
({rule head) < — (rule body)), adapted for tree structured data. The 
counterpart of datalog predicates are tree expressions, defined and 
explained below. 

We explain the mapping expression language next. The rules are 
made up of atoms of the form Tag — > id, where Tag is a tag or a 
tag variable and id is the id associated with a node with this tag. 
Here, id may be a variable or any term of the form f($vi, $v n ), 
for some variables $v-i and some Skolem function /. Atoms can be 
nested inside other atoms, thus expressing nesting, while a comma- 
separated list of atoms is used for expressing the subelements of a 
given element. Attributes are preceded with a '@'. Before we ex- 
plain the meaning of the rules, it's important to bear in mind that 
the rules and the mapping are not intended for physically trans- 



Figure 2: Mapping Rules between the schemas in Figure 1. 

forming data from one source's schema to another. As pointed out 
in [2], they are rather intended for expressing the semantics of data 
exchange - if data were to be exchanged from source 1 to 2, how 
would it correspond to the schema of source 2. 

Atoms can be nested to form tree expressions. Tree expressions 
are either atoms (t — > i) or are of the form t — > i\TE\, TEj,], 
where is an atom and TEi are tree expressions. In Fig- 

ure 2, each rule head is a tree expression while the rule body is 
a conjunction of tree expressions and built-in predicates (=, >, 
etc.). Rule 1 says corresponding to the (unique) root of the MonG 
source, there is a (unique) root in MasG. The uniqueness of the 
latter follows from applying the Skolem function fl to the node 
id associated with the former's root, which is unique. Similarly, 
there is a unique Admission node in MasG, and we have used 
again the root of MonG as the argument of the Skolem function 
/2 for capturing this uniqueness. The rule body binds the variable 
$AP to the problem of a patient at admission time. $AP/text() 
extracts the text value associated with node $AP. This value 
(which may have a value from the same domain as 'Pulmonary' 
and 'Coronary'") is used to form the tag of a new node, whose id 
is f3($AP/text(), $ID, $M, $AD, $DD, $N), i.e., it is a func- 
tion of the patient's admission time problem ($AP/text()), id 
($ID), insurance policy (or medicare) number ($M), admission 
and discharge dates (SAD, SDD), and name ($N). Note that 
the arguments of the Skolem function /3 are exactly the single- 
valued subelements of the Pulmonary and Coronary elements 
in MasG. 3 We do not assume any knowledge of keys in this paper. 
Note that patient id, name, policy number, admission and discharge 
dates are all matched to their counterparts in MasG. 



2 The value need not be one of these. 

3 Optional elements are handled using marked null values. 



Input: A DTD A. 

Output: A set of groups of nodes from the graph of A. 

1 . Start from the root. 

If there are multiple outgoing edges, mark the root as P; 
Else follow the single path (no restriction on edge labels) 

until it reaches a node that has multiple outgoing edges; 

if there is none, return all nodes in A as one group; 

else mark the node found as P; 

2. Start from node P: 

Follow each outgoing edge, and stop when we visit an edge 

with label ' * ' or ' + ' ; 
Mark the node that this edge points to as a "stop node"; 
If there are multiple edges with label '*' or ' + ' 

pointing into nodes in a box 

(e.g. Pulmonary, Coronary in Figure 1) 

Mark the box as the stop node; 

3. If there is at most one stop node found in A, 

Group the descendants of current P node and all 
the nodes on the path from root to P via previously 
recursively marked P nodes (if any); 

Else 

For each stop node S { 

If there are multiple outgoing edges, 
mark this node as P; 

Repeat (2) and (3) to find its stop nodes until it 
reaches leaf nodes or find at most one stop node; 

Else 

Group the descendents of current stop node S, 
and all the nodes on the path from root to S 
via previous recursively marked P nodes; } 

4. Group the rest of the nodes and their ancestor nodes together. 



Rule 2 maps the patient history consisting of Problems and their 
Dates of occurrence (nested in MonG through Hist/Event) 
to Symptom/Desc and Symptom/Date in MasG. Note that in 
MasG, the Symptom elements are nested inside a Progress 
element, which has as its id a function of the patient ID (via 
OPatRef), i.e., f2($PR), $PR = $ID. Thus, there is one 
Progress element per patient. Consequently, Symptoms are 
grouped by patient ID. The node ID f3($EP, $ED) used for 
Symptom elements shows that for each occurrence of a problem 
for a given patient, a separate Symptom element is created. 

Rule 3 maps treatment information from MonG to MasG. 
Progress elements are created with id /2($Pi?) just as they are 
in rule 2. Note that the use of the node id fZ(%TDate, %TDesc) 
for Treatment ensures that for every treatment on any date ad- 
ministered to a given patient, the corresponding Treatment ele- 
ment is nested inside the Progress element associated with the 
patient. 

Node id's play a key role: for instance, Progress elements are 
created by rules 2 and 3 independently. Whenever the id of a 
Progress node created by rule 2 matches the one created by 
rule 3, they refer to one and the same node. For instance, sup- 
pose 'p5' is the ID value of a patient. Then the subtree rooted at the 
Progress node /2(p5) created by rule 2 and the subtree rooted 
at the Progress node /2(p5) created by rule 3 are both glued 
at the node /2(p5). More generally, whenever subtrees are cre- 
ated by applications of the same or different rules, conceptually all 
these subtrees are glued together at nodes having a common node 
id. This ensures that the pieces "computed" by rules are correctly 
glued together. 

Finally, rule 4 maps @PatRef attribute in MonG to @PatRef 
attribute in MasG, while equating the @ID, SPatRef attributes in 
the MasG DTD. 

So far, we explained meaning of mapping rules. A main challenge 
is their automatic derivation from user supplied informal corre- 
spondences, and is addressed in Section 3, where we also briefly 
discuss the significance of the class of transformations captured 
by the rules, from an algebraic perspective. The next challenge 
is translating queries posed against one peer schema so they can be 
answered from any source. This is dealt with in Section 4. Finally, 
the scalability of the framework for a real P2P DBMS setting is 
empirically established in Section 5. 

3. INFERRING MAPPING RULES FROM 
ARROWS 

In this section, we address the following question. Given a pair 
of DTDs, represented as graphs, and a set of arrows/boxes relating 
nodes across the graphs, how can we automatically infer a set of 
rules for mapping instances of one DTD into instances of the other. 
First, observe that owing to missing elements, we cannot always 
map a valid instance of one DTD into a valid instance of another. 
E.g., in Figure 1, there are no counterparts for the nodes Doc and 
InsName in the other DTDs. Recall that the purpose of mapping 
is not for physically transforming data across schemas, but rather to 
capture the correct semantics of data exchange. In this section, we 
develop an automatic mapping rule inference algorithm. We will 
use our running example (Figure 1) to illustrate the algorithm. 

Suppose we wish to infer mapping rules from a DTD Ai (call it 
source) to another A2 (call it target) based on given correspon- 



Figure 3: Algorithm for Detecting Groups in DTDs. 



dences. The algorithm consists of the following main modules. (1) 
Determine groups of nodes in the two DTDs such that each group 
intuitively captures some "unit" of information. (2) For each group, 
if the group induces a DAG in the original DTD graph, convert it 
into a tree. Then construct a tree expression describing a unit of 
information following the hierarchical structure of the group. (3) 
For each target group, identify all minimal sets of source groups 
necessary to populate information into the target tree expression 
structure and construct the rules. We explain these steps below. 

3.1 Detecting Groups 

First, we need to determine groups of nodes in both DTDs to which 
variables can be bound in a single tree expression in a rule body or 
a rule head. To appreciate this, consider mapping instances of Fig- 
ure 1(a) to those of Figure 1(b). Suppose we write a rule of the form 

(whatever) < MonGenHosp — > $MoraG[Patient — ► $P[...], 

Admission — > $A[...]]. Then we create the objects as per 
the rule head, for each combination of patient and admis- 
sion. Thus, the multiplicity of the elements in the original 
database will not be preserved by this rule. This problem will 
be solved if we write mappings for the following groups of 
nodes separately: {MonGenHosp, Patient, @ID, MedCr#, Name} 
and {MonGenHosp, Admission, Problem, AdmDate, DisDate, 
©PatRef }. The module for determining groups is given in Fig- 
ure 3. Our algorithm can actually deal with disjunction in DTDs, 
but we omit disjunction for simplicity. We do not consider cyclic 
DTDs and do not consider order in XML. We next explain the 
group detection algorithm (Figure 3). Figure 4 shows the groups 
created. 

Suppose we apply the group detection algorithm on the MonG 
DTD of Figure 1(a). Following step 1, we first visit MonGenHosp 
and mark it as P (it has multiple outgoing edges). Then, per Step 2, 



we visit Patient and (eventually) Admission, and mark them 
as stop nodes (since the edge labels are *). According to step 
2, Patient is again (recursively) marked as P (> 1 outgoing 
edge) and its associated stop nodes Event and Treat are found, 
each of which would again recursively be marked as P. Applying 
step 3, since there are no stop nodes reachable from Event, we 
group all descendants of Event, add all nodes on the path from 
MonGenHosp (root) to Event and create the group sgi in Fig- 
ure 4, where we have qualified the shared elements Problem and 
Date (i.e., EProblem and EDate) to distinguish them from ad- 
mission problem and admission date. In a similar fashion, we will 
form the groups sg2 and sgs. Finally, by step 4, we will form 
the group sg4, since @ID, MedCr#, Name are the left-over at- 
tributes and elements for the recursive call on Patient marked 
as a P node. We invite the reader to verify that the algorithm will 
form tgi, tg?., tgz and tg^ (as shown in Figure 4) on the MasG 
DTD in Figure 1(b). 



sgi — {MonGenHosp, Patient, Hist, Event, EProblem, EDate} 

sg2 — {MonGenHosp, Patient, Treat, TDate,TDesc, Doc} 

sg$ — {MonGenHosp, Admission, Admission_Problem, AdmDate 

DisDate. OPatRef } 
sg 4 — {MonGenHosp, Patient, @ID,MedCr#, Name} 

tgi — {MassGeneral, Admission, Pulmonary, Coronary, @ID, InsName, 

Policy^, Enter, Leave, Patient} 
tg-2 — {MassGeneral, Progress, Symptom, SDate, SDesc} 
tgs — {MassGeneral, Progress, Treatment, TDate, TDesc} 
tgi — {MassGeneral, Progress, @PatRef}. 

Pairs of Groups Connected by Arrows: 

(•5S3 , tgi ) , (sgt , tgi ) , (sgi , tg 2 ) , (sg 2 ,tg 3 ), (sg 3 ,tg 4 ). 



Figure 4: Example of Group Determination for Figure 1. 

If we want to write a mapping from one DTD (call it the source) 
to the other (call it the target), e.g., from MonG to MasG, then we 
consider every target DTD group and write a rule for it by exam- 
ining those groups of the source DTD that are connected to it by 
arrows. In our running example, the pairs of groups connected by 
arrows are shown in Figure 4. 

3.2 Generating Tree Expressions 

The next step is to write tree expressions using the groups identi- 
fied. Before doing so, for each group, we examine the subgraph 
of the original DTD graph induced by the nodes in the group (not 
counting ID — » IDREF(S) edges). Our group formation algorithm 
always ensures these subgraphs are connected. If furthermore, the 
subgraph is a tree, we are ready to write the tree expression for 
that group. If it is a DAG, then we replicate each shared node 
(recursively) as many times as necessary to create a tree struc- 
ture. E.g., consider a standard DTD for books: bib — > book*, 
book — > title author pub, author — t name, pub — » name. It is 
trivial to see that this whole DTD would be flagged as a single 
group by our algorithm. The structure of this DTD graph is a DAG. 
We replicate the name node twice - once for author and once 
for pub to create a tree structure. An exception is when the DAG 
structure is the result of multiple nodes in a box sharing the same 
substructure. E.g., in the target DTD MasG, Pulmonary, Coronary, 
etc., all share the same substructure and they are in a box. In this 
case, we will not replicate the shared elements. Instead, the use of 
a tag variable will deal with this correctly. 

For each source group, the tree expression is written by essen- 
tially following the recursive tree structure, using [...] to capture 
the nesting. For every node in the group, we write the expression 



Input: source groups, and target groups 
Output: a set of mapping rules 
For every target group tg 

Let {sgi , . . . , sgj } be the set of source groups connected to 

tg by arrows 

Let TE(tg) be the tree expression for group tg 

1. Stall with the rule skeleton 

TE(tg)A—TE(e gi ),..., TE(sg 3 ) 
Fill in the variables corresponding to leaf positions in TE(tg) 
based on the arrows incident on the leaf elements of tg 

2. For root and each of its descendants via only single-valued edges 

Assign their ids as distinct Skolem functions of the root 
variable in the source 

3. For each internal node inode 

Assign its id as a distinct Skolem function of the variables 

associated with all its single-valued children 

If any of its single-valued children sc does not belong to tg 

Trace the source group sg p with an arrow pointing to sc 

Add TE(sg p ) in the rule body: 

TE(tg)^TE(sg z ), TE{sg 3 ),TE{sg p ) 

Let scref denote the corresponding element of sc in the 

source schema 

If scref points to a node in the source schema scid and the 
source group sg q that scid belongs to is not in {sgi . .... sgj } 
Add TE(sg q ) in the rule body: 

TE(tg)< TE(s 9l ), TE(sg 3 ), TE(sg p ) .TE(sg q ) 

Equate the variable scref binds to (Sscre/) with the 
variable scid binds to ($scid): $scref=$scid/text() 
(if scid is an attribute, then use $scref=$scid instead) 
Add %scref as an argument to the Skolem function 
on the RHS of inode 



Figure 5: Rule Construction Algorithm. 

Tag — > $var, where Tag is the tag name of the node and $var is a 
new variable. If the node is inside a box (e.g., like Pulmonary in the 
MasG DTD), then we use a tag variable and write $Tag — > $var. 
Group sg2 induces a tree so we can immediately write its tree ex- 
pression as MonGenHosp — > $A/g/i[Patient — > $P[Treat — > $T 
[Date -> $TDate, Desc -v %TDesc, Doc -> $Doc]]]. All the 
source groups happen to induce trees. 

For target groups, we follow the same procedure, replicating nodes 
if necessary to convert any DAGs induced by a group into a tree. 
Once we have trees, we write tree expressions with them. However, 
there is a major difference w.r.t. source groups, i.e., we do not know 
the node id's to be used in the generated tree expressions. Instead, 
we write the tree expression as a skeleton, leaving the node id's 
as '??' for now. As an example, the tree expression for tgi would 
be MassGeneral -> ?? [Admission -> ?? [$Tag -> ?? [@ID -» ?? 
, Patient — > ??]]]. The '??' will be filled in when we write the 
mapping rules. We drop a leaf node from consideration if there is 
no counterpart in the source schema. E.g., Doc is one such node. 
The module for creating trees and for writing tree expressions is 
straightforward and is not shown. 

3.3 Generating Mapping Rules 

The last step is writing the mapping rules. We present the rule 
construction algorithm in Figure 5. 

Consider each target group tg. Let {sgi, sgj} be the set 
of source groups connected by arrows to tg. Let TE(g) de- 
note the tree expression for group g. Applying step 1, we start 
with the rule skeleton TE(tg)< — TE{sgi), TE(sgj). Based 
on the arrows incident on the leaf elements of tg, we fill in 
the variables corresponding to leaf positions in TE(tg), i.e., the 
right-hand-side of atoms corresponding to leaf nodes. E.g., for 
tg2, we would start with TE{tg'z) < — TE{sgi). The rule body 



only contains TE(sg 1 ) since that is the only source group con- 
nected to tg-2 by arrows (see Figure 4). Based on the arrows, 
we can fill in the right-hand-sides of Date and Desc in the 
rule head as %ED and %EP, respectively: MassGeneral — > ?? 

[Progress ?? [Symptom -> ?? [Date -> ??, Desc -> ??]]] < 

MonGenHosp -> %M [Patient -> $P [Hist -v %H [Event -v %E 
[EProblem -> %EP, EDate -> $SD]]]]. 

Next, according to step 2, for root MassGeneral and its single- 
valued child Admission, we assign their ids as distinct Skolem 
functions of the root variable in the source (%Mgh). 

Then we apply step 3: for each internal node, we assign its id 
as a distinct Skolem function of the variables associated with all 
its single-valued children. E.g., for the RHS of Symptom, we 
would use the Skolem function f3($EP, $ED). For Progress, 
its (only) single-valued child is @PatRef , which does not belong 
to tgi. By following the arrow incident on SPatRef we trace 
source group sgz, so we introduce TE(sgs) in the rule body. Now, 
SPatRef in the source schema points to the ID attribute @ID of 
Patient, an attribute that does not belong to sgi or s<?3. How- 
ever, @ ID belongs to sgn so we also add TE{sgi) to the above rule 
body. We equate the variable associated with the SID attribute in 
TE(sg4,) with the variable associated with SPatRef in sgi. At 
this point, the rule looks as follows: 

MassGeneral — > fl(SMgh) [Progress — > ?? 
[Symptom -> f3($ED, SEP) 
[Date -> SED, Desc -» SEP]]] 

MonGenHosp -» SM [Patient -> $P [Hist -> SH 

[Event -» SE [Problem -» SEP, 

Date ^ $££>]]]], 
MonGenHosp — > SM [Admission — > SA 

[Problem — > SAP, AdmDate -> SAD, 

DisDate -» SDD, ©PatRef -> SPR]], 
MonGenHosp -» SM [Patient -> SP 

[@ID^$7,Name^$AT, 

MedCr# -> SMC]], SPR = SI. 

Now, for the RHS of Progress, we can assign the Skolem func- 
tion f2($PR). We then refine the rule by identifying nodes/paths 
shared between two or more tree expressions in the body. This 
would yield rule 2 in Figure 2. Note that the generated rules are 
always safe - all variables in the head appear in the body. The 
main contribution of this section is the automatic inference of map- 
ping rules that transform tree database instances of one schema into 
those of another. 

We next briefly address the question, what is significant or 
fundamental about the class of transformations that are captured 
by the rules? For lack of space, we merely sketch this idea 
and refer the reader to ftp://ftp.cs.ubc.ca/~laks 
/algebraicTransf ormations4Heptox . pdf for more 
details. The key idea is that the rules capture a class of database 
tree transformations that are expressible using the operators 
unnest/nest (similar to those for nested relations), flip/flop (which 
basically change nesting orders in the schema), and merge/split 
(which have a flavor of grouping and "ungrouping" a set of 
nodes. It can be shown that the rules capture precisely the class 
of transformations expressible using these operators together with 
a few additional operators like node addition/deletion and tag 
modification, added for completeness purposes. 

A comment about optional single valued elements which may form 
arguments of Skolem functions. We can model their optionality 



using distinct marked null values. The details will appear in the 
full paper. 

4. QUERY TRANSLATION 

In this section, we address two questions. (1) Suppose we have a 
pair of peer XML database sources pi, with DTDs A; and under- 
lying database instances Di, i — 1,2. Suppose we issue a query 
Q against the DTD of pi ($2). What does it mean for Q to be 
answered using the database of P2 (pi)? (2) Can we translate the 
query Q against the other peer's DTD (say A2) such that the trans- 
lated query Q' against A2 evaluated on the database atj>2 will yield 
the correct answers w.r.t. the semantics captured by the answer to 
question (1)? We also wish to do the translation efficiently. 

4.1 Query Translation Semantics 

Suppose we have the peers, DTDs, and database instances as de- 
fined above. Suppose we have mapping rules /1 mapping database 
instances of Ai to those of A2, i.e., /1 : Ai — ► A2. Now, a query 
Q can be posed against Ai or against A2. The following definition 
captures the correct semantics of query translation for these two 
cases. Let inst(A) denote the set of database instances of A. 

Definition 1. [Semantics] Suppose Qi is a query posed 
against A,, i = 1,2. Let Q\ denote a translation of Qi against 
Aj, j 7^ i. Then Q\ is correct provided VZ?i € mst(Ai) : 
Qt{Di) — Q2(fJ.(Di)). The translation Q\ is correct provided 
VD 2 G mst{A 2 ) : Q\(D 2 ) = Ruf !/l(D f )= n 2 QiPft). ■ 

The translation Q\ is correct provided evaluating Q2 on the trans- 
formed instance /i(-Di) and evaluating Q\ on D\ both yield the 
same results, for all Di £ inst(Ai). Note that in this case, the 
direction of translation is against that of the mapping /j,. This is 
the easy direction. Consider translating a query Qi posed against 
Ai to the DTD A2 of peer p 2 - This is aligned with the direction 
of the mapping /i. The complication here is the mapping /i that 
transforms instances of Ai to those of A2 may not be invertible. In 
this case, we require that no matter what instance D\ € inst(Ai) 
was transformed by fi to D2 (i.e., fJ,(Di) = D 2 ) the answer to 
Q\ on D 2 should coincide with the answer to Qi on D\. This 
is captured by taking the intersection of the latter answers over all 
possible preimages for which D 2 = fx(D^). 

Note that owing to schema discrepancies between Ai and A 2 , the 
output of Q2(l-i(Di)) would "conform" to the schema A2. What 
can we say about Q 2 (Di)l Note that XQuery permits restructuring 
of output so this is not an issue. A similar comment applies to the 
translation Q\ of Q\. One of the main contributions of this paper 
is that our query translation algorithm, given in the next section, is 
correct in the sense defined above, regardless of the direction of the 
mapping rules. 

4.2 Query Translation Algorithm 

XQuery fragment considered: The fragment of XQuery we con- 
sider corresponds to queries expressible as joins of tree patterns 
(TP) [1 J, where the return arguments correspond to leaf nodes of 
the database. We note that even for this simple fragment of XQuery, 
query translation is far from trivial. We briefly review tree patterns 
next. 

A tree pattern is a rooted tree with child and descendant edges, and 
with nodes labeled by variables. Each node variable may be addi- 
tionally constrained to have a specific tag (e.g., name(Sx) = book) 
and (in case of leaf elements) possibly constrained on its value (e.g., 



Sy/text() = "123" or $x/text() > $y/text()). Figure 6(b) shows an 
example in which some constraints of both types are specified. In- 
tuitively, the semantics of a TP is that its nodes are matched to the 
data nodes in an input XML database while preserving tags, edge 
relationships, and any constraints on node contents. Each match 
leads to an answer that consists of the data node that matches a 
specially marked distinguished node of the TP. E.g., in Figure 6(b) 
the answer should contain Name nodes which corresponds to the 
output node marked with a dashed box. As a convention, if no ex- 
plicit distinguished node is marked in an example, we intend that 
all leaf nodes of the TP are distinguished. 

4.2. 1 Translating Tree Patterns 

Let us now return to query translation. Naturally, the details of 
the translation vary a little depending on whether we translate in 
the direction of the mapping rules or against it. Let us first consider 
translating in the direction of the mapping rules (i.e., from the body 
of each rule to its head). So, we want to translate a query Qi on 
Ai to one on A2. As an overview, we represent a query as (a 
join of) one or more tree patterns and translate them one by one. 
The translation consists of three phases: (1) An expansion phase, 
where we expand the TP by adding more nodes and edges so it 
can be matched to a rule body. It may also involve "unwinding" 
a descendant edge to spell out the intermediate nodes on a path. 
Essentially, the rule body (i.e., the tree expression in the body) is the 
expanded TP. Expansion requires finding a substitution that relates 
variables in the rule body to those in the TP. (2) A translation phase, 
where we "apply" the rule and generate the instance of the rule 
head corresponding to the substitution used. (3) A contraction (or 
shrinking) phase, where certain nodes are identified as redundant 
and are eliminated from the translated TP. We explain these steps 
in detail below. 



SPiob SKefVal 

$Adm.tag= Admission & 

SProb.tag=Problem & 
$Prob/text()-"Coronary" 




- Date & $D/text() = "1 2/25/2003" 

(5) 



Figure 6: Join of two Tree Patterns. 

Expansion: Let t be a TP and r : h r < — b r be a mapping rule. Re- 
call that the body b r consists of tree expressions possibly together 
with some comparison predicates (see Figure 2, e.g.). We first find 
a matching of t to b r , which is a substitution that maps variables in 
b r to those in t. This substitution (mapping) may be partial in that 
b r may contain components which have no counterpart in t. E.g., 
consider Figure 6(a). Ignore the join condition for now. Let us try 
to match this TP to the body of rule 1 in Figure 2. The correspond- 
ing expansion is shown in Figure 7(a). Note that the expanded TP 
is just the body of rule 1. The part of the expanded TP that was 
originally present in the TP (before expansion) is shown in green 
while the rest (i.e., the new edges coming from expansion) is shown 
in red (colors are represented with line styles as in the legenda). We 
refer to nodes that were not part of the original TP as dummy nodes. 
In Figure 7(a), nodes with no incident green edges are precisely the 
dummy nodes (e.g., $Mgh, $P, $DD, etc.). 

Translation: Next, we translate the expanded TP by apply- 
ing the rule to it. The correspondence between the variables 
in the original TP and those in the expanded TP (i.e., the rule 
body) is kept track of by means of a substitution between the 
two. Figure 7(a) shows the substitution for the TP in question 



as {%A -> %Adm, %AP -> %Prob, %PR -> %Ref Val}. Using this 
substitution, original query constraints are propagated through the 
translation. In our example, the translated query resulting from ap- 
plying rule 1 to the expanded TP above is shown in Figure 7(c) 
(ignore the blue edge for now). For readability, all tag constraints 
are shown concisely by writing the tags next to the appropriate 
nodes. Note how the constraint $Prob/text() — "Coronary" is 
propagated via the substitution as $AP/text() — "Coronary" . 
Additionally, the condition $PR = $ID in the body of rule 1 is 
used to infer that the attribute child @ID of the node %AP/text{) 
in Figure 7(c) corresponds to the attribute child @PatRef of 
the node $A in Figure 7(a). At this point, the translated query 
contains Skolem functions at certain nodes. By keeping track 
of the correspondences between the dummy nodes in the ex- 
panded TP (Figure 7(a)) and the translated query, we can iden- 
tify nodes that can be eliminated from this query. Specifically, 
in the expanded TP, only nodes $AP, $PR, and $A were non- 
dummy nodes. The corresponding nodes in the translated query are 
fS($AP/text(), $ID, SM, $AD, $DD, $N) (a Skolem function 
whose arguments include $AP/textQ), and $ID (which corre- 
sponds to $PR). Note that there is no counterpart for $A in Fig- 
ure 7(c). So, all other nodes in the translated query are dummy 
nodes and can be dropped. More precisely, if a translated query 
node corresponds to a non-dummy node from the expanded TP or 
is a Skolem function one of whose arguments corresponds to non- 
dummy node, then it is non-dummy. Otherwise, it is dummy. 

Contraction: Having identified dummy nodes in the translated 
query, we try to drop them. For leaf dummy nodes, this is trivial and 
they are always dropped, while preserving query equivalence. An 
internal dummy node can be dropped provided dropping it and con- 
necting its parent and children with descendant axis ('//') will not 
change the corresponding set of DTD paths connecting them. So, 
whenever an internal dummy node is dropped, its parent and chil- 
dren are connected by a descendant edge. In Figure 7(c), all nodes 
with no incident green edges are dummy and can be dropped. Note 
that the set of paths connecting the remaining nodes remains un- 
changed according to the DTD A2. Each remaining (non-dummy) 
node is replaced with its corresponding tag. If the tag is an un- 
constrained variable, it is replaced by the wildcard '*'. If it is a 
constrained variable, it is replaced by each valid tag satisfying the 
constraint. Whenever internal dummy nodes are dropped, the cor- 
responding child edges are replaced by descendant edges. Doing 
this to the translated query in Figure 7(c) yields the final contracted 
(shrunk) translated query in Figure 7(d). So, at this point, the final 
query is //Coronary/@ID. 
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Figure 7: Translation of Join of TPs from Figure 6. 



It is easy to verify that when query translation is applied to the TP 
in Figure 6(b), using the body of rule 1 in Figure 2, we will get 
the expanded TP in Figure 7(b), the translated query in Figure 7(c) 
(with both green and blue edges considered), and finally the con- 
tracted translated query in Figure 7(e) (with both green and blue 
edges considered). 

4.2.2 Translating Joins of Tree Patterns 

Now, consider the query in Figure 7(a)-(b), including the join con- 
dition SPR — SID. As illustrated above, Figure 7(a) leads to the 
contracted translated query //Coronary/@lD while Figure 7(b) 
leads to the contracted translated query / /Coronary [@ ID] /Name. 
From the join condition, we deduce that the @ID node in both 
queries denote the same value and hence their parent Coronary 
nodes must be identical. Based on this, we "stitch" the two trans- 
lated queries together by identifying their Coronary nodes and 
their @id nodes. So, the final contracted translated TP is isomor- 
phic to the one shown in Figure 7(e) (with both green and blue 
edges considered). 

4.2.3 An XQuery Example 

Let us illustrate how our translation algorithm developed so far 
can handle a simple fragment of XQuery. This example illustrates 
translation in the direction of the mapping rules. 



EXAMPLE 1. [XQuery Forward] : Consider the XQuery 
query: "Find all patients with Admission/Problem = 'Coronary' 
whose Treatment started Dec, 25th 2003", expressed as: 



Ql: FOR $A IN //Admission, 

$P IN //Patient [@ID=$A/@PatRef] 
WHERE $A/Problem="Coronary" AND 

$P/Treat/Date=" 12/2 5/2 003" 
RETURN {$P/Name} 

This query can be represented as the join of two tree patterns, as 
shown in Figure 6(a)-(b). In the previous section, we discussed 
how using rule 1 of Figure 2, this join query can be translated to 
the tree pattern in Figure 7(e). But that is not yet the translation of 
query Ql itself: we have to consider translation via other mapping 
rules in Figure 2 as well. A quick inspection reveals rule 2 is not 
relevant to the query as the rule heads do not contain variables cor- 
responding to the variables in the query of Figure 6. Consider the 
translation via rule 3. The expanded TP is shown in Figure 8(a). Its 
corresponding translated query is shown in Figure 8(b). The trans- 
lation w.r.t. rule 4 is shown in Figure 8(c) (expanded TP omitted). 
The contracted translated queries are shown in Figures 8(d). An 
important point to note is that based on the equality between SID 
and SPR, the two / /Progress nodes (which respectively corre- 
spond to f2(SID) and f2($PR), with SID = SPR) have been 
merged. 

Finally, in Figure 8(e), we combine the translated query obtained 
via rules 1, 3, and 4 (see also Figure 7(e) and Figure 8(d)). The key 
point to notice is that since the variables SID and SPR have been 
equated in the mapping rules, we can infer that the attributes @ID 
and @PatRef must be equal. This is again the join of two TPs and 
the corresponding XQuery is given in Figure 8(f). ■ 



When Q is a query expressed against the DTD A2, the key intu- 
ition for query translation is to essentially follow the mapping rule 
in the reverse direction, i.e., from the head to the body. This has 
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Figure 8: Translation of Ql completed: tag constraints sup- 
pressed for avoiding clutter; red = solid, thick; green = dashed, 
thick; blue = dashed & dotted, thick. 

resemblances to query folding and answering queries using views 
[15]. However, the presence of Skolem functions greatly simpli- 
fies this process. The reason is that the node id's act as a "glue" 
suggesting which subelement pieces should be associated together. 
Consequently, they drive exactly which mapping rule bodies we 
need to "join" together to rewrite the given query. We illustrate this 
with an example. 



Example 2. [XQuery Backward] : Consider the XQuery 
query: "Find symptoms of patients admitted with 'Pulmonary' (ail- 
ment)". 



Q2 : FOR $P IN //Progress, 
$Y IN //Pulmonary 
WHERE $Y/@ID=$P/@PatRef 
RETURN {$P/Symptom/Desc} 

Figure 9 shows the join of TPs corresponding to this query. The 
next step is to translate each TP by matching it to each rule head. 
It is easy to see that only the heads of 1, 2, 4 can be matched to 
the query (partially). 4 The expanded TPs are obtained by match- 
ing the TPs against each of these rule heads and are shown in 
Figure 10(a)-(c), where to minimize clutter, we do not show tag 
constraints nor the substitutions between query variables and the 
terms in the rule heads. In essence, as for the forward direc- 
tion of translation, each expanded TP is a rule head expressed 
as a tree expression. From the names of variables and tags the 
substitution between query variables and rule variables should be 
implicitly clear. We also mark the variables that appeared in 
the original query (shown via distinctly colored edges in the fig- 
ure). E.g., in Figure 10(a)-(c), we know SPulm is associated 
with the node f 3 (SAP/text(), SID, $M, SN, $AD, SDD), $ID 
with the node SID, SProg with node f2(SPR), SSymp with node 
/ 3 (SEP, SED), SDesc with node SDesc, $Prog also with the node 
f 2 {SID), and finally SPR with node SPR. 

Next, each expanded TP is replaced by the tree expression in the 
corresponding rule body (Figure 10(d)-(f)). In doing so, we again 
keep track of variables mentioned in the original query. For lack 
of space, we give just two examples of this. Query variable SPulm 
corresponds to each of the nodes labeled SAP in Figure 10(d)-(f) 
and query variable SDesc corresponds to the node labeled SEP in 
Figure 10(e). The reader should be able to work out the other cor- 
respondences with a modest effort. 



4 Actually, in the head of rule 3 just Progress can be matched to 
the query, but this is subsumed by matches to other rule heads and 
is redundant. 



The next step is to drop dummy nodes. The idea is very similar to 
that adopted for the forward direction of translation and is not elab- 
orated further. Dropping of dummy nodes generates simplified but 
equivalent TPs. Nodes in different TPs that correspond to the same 
query variable are stitched together. E.g., the $A and $PR nodes in 
Figure 10(e) and (f) are merged. The final result of merging nodes 
is shown in Figure 10(g), which is actually a join of two TPs. The 
TPs are shown more concisely by writing the tags directly in place 
of the variables that are constrained by those tags. The join of TPs 
actually corresponds to the following XQuery statement: 



FOR $P IN //Patient, 

$A IN //Admission [@ID=$P/@ID] 
WHERE $A/Problem=' Pulmonary' 
RETURN {$P/Event} 




Figure 9: Join of TPs corresponding to Q2. 
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FigUre 10: Steps in the Translation of Q2; red = solid, thick; 
green = dashed, thick; blue = dashed & dotted, thick; cyan = 
dashed thick. 

The algorithm for query translation is given in Figure 1 1 , and it 
closely follows the major steps outlined in Examples 1 and 2. Note 
that the algorithm shown handles one TP at a time. Handling joins 
of TPs follows the same general steps as discussed in the examples. 
We have the following result concerning the correctness of the al- 
gorithm w.r.t. query answering semantics defined in Definition 1. 

Theorem 1. [Correctness of query translation] : For the 

fragment of XQuery defined in Section 4.2, the query translation 
algorithm in Figure 1 1 is correct w.r.t. the semantics of query an- 
swering defined in Definition 1. ■ 

For space reasons, we omit the proof. The proof is based on an 
adaptation of the well-known chase proof procedure. Before leav- 
ing this section, we note that our query translation algorithm is cur- 
rently able to handle limited forms of nested XQuery expressions, 
as long as the query can be represented using TPs (with joins). 



Input: a tree pattern query Qi, mapping rule set from — > A 2 
Output: one translated query Q\ 
If Qi is against Ai 

1. For each mapping rule fij 

a. Find all substitutions and expand Qi to Qfj by matching the rule 
body rbj to the query pattern Qi 

For each node rbj and not in Qi , mark it as dummy 
If it is a distinguished node in Qi 

mark it as distinguished in the expanded query Q| . 

b. Translate each Q|. into Q** by applying fij 

Mark dummy and distinguished nodes accordingly 

c. Obtain contractions Q* . of previous translations by recursively remo- 
ving leaf dummy nodes and appropriate internal dummy nodes in Q\* 

2. Stitch together all Q\- to obtain resulting query Q\. Nodes with same 

query variable or Skolem terms get stitched by looking at the cor- 
responding substitutions. 
Return query corresponding to Q t 
If Qi is against A 2 

l'.For each mapping rule fij 

a' .Find all substitutions and expand Qi to Q\^ by matching the 
rule head rhj to the query pattern Qi and by looking up the 
correspondence in the rule body rbj 

For each node in the rule head rhj and not in query 
pattern Qi, mark it as dummy 
If it is a distinguished node in Qi 
mark it as distinguished in the expanded query Q^- 
b\ Translate each Q|- into Q** by projecting every variable of 
rule head rhj to rule body rbj 

Mark dummy and output node accordingly 
c'. Obtain contractions Q* . of previous translations by: 
cl) removing existential variables not appearing in rhj 
c2) recursively removing leaf dummy nodes and 
appropriate internal dummy nodes in Q^* 
2'. Stitch together nodes in all Q|j that correspond to the same query variable 

(via substitution and comparison predicates) to obtain resulting query Q\ 
Return query corresponding to Q\ 



Figure 11: Query Translation Algorithm 



5. EXPERIMENTS 

Implementation and Setup: In this section, we examine how the 
query translation algorithm impacts the query performance and the 
scalability of HepTox. Using the mapping rules discussed in Sec- 
tion 3, each peer maps to a set of acquaintances. Transitive map- 
pings produces semantic paths, and each peer is subsequently se- 
mantically connected to every other node. By means of a prelim- 
inary experiment, we realized that in order to obtain a reasonably 
small number of hops in network with number of peers <= 128, 
we have to consider sets of acquaintances having at least 4 peers. 
Henceforth, we assume to have such sets of acquaintances for all 
experiments, which guarantee an average number of hops equal to 
5. 

Our aim is to observe how our approach behaves in a network of 
XML database systems having heterogeneous schemas. Hence, we 
derived 9 restructured variations of the XMark DTD [23], corre- 
sponding to the various transformations expressible in our mapping 
rule language, to produce 10 different schemas randomly scattered 
across the network. Detailed description of the schemas can be 
found on the HepToX home page [12]. We modified the XMark 
xmlgen code accordingly to generate the data sets corresponding 
to the schemas above, with sizes ranging 10MB -100MB. We use 
Emulab [8], a network emulation testbed, to emulate a realistic P2P 
database. Emulab consists of a collection of PCs, whose network 
delay and bandwidth can be set at wish. We chose a 70ms delay and 
a 50MB bandwidth to simulate as much as possible the geograph- 
ical networks behavior. We could get 30 real machines for emula- 
tion. As network protocol among emulab machines, we mounted 



FreePastry [19], which offers a scalable and efficient P2P routing 
algorithm. Its O(logN) routing complexity, and O(logN) routing ta- 
ble size is at least as efficient as other similar projects such as CAN, 
Tapestry and Chord. Heptox integrates FreePastry with QIZX [22], 
which is apparently the fastest available open-source XQuery en- 
gine. We used FreePastry vs. 1.3. 2 and QIZX vs.0.4pl, respectively. 

Guiding Principles: The experiments are conducted according to 
the following guidelines: (i) peers gets created within the Pastry 
network, each peer owning an independent dataset, stored in its 
own QIZX query engine; ( ii) each peer selects other peers as its ac- 
quaintances, and these acquaintances in turn see this peer as their 
acquaintance; (Hi) then a peer initiates a query, translates the query 
to the schema of each acquaintance, using our algorithm, and sends 
the translated query to that acquaintance; (iv) these acquaintances 
in turn translate the query and send it to their acquaintances, which 
process the query and return all the results backward to the origi- 
nating peer; (v) forwarding of queries by a peer stops as soon as the 
peer realizes it received an already processed query request. 

Scalability of HepToX: The first experiment is devoted to measur- 
ing the efficiency of the Heptox query translation algorithm (Fig- 
ure 11). 
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Table 1: AQTT (Average Query Translation Time in ms); 
MQTT (Maximum Query Translation Time in ms); ASPL (Av- 
erage Semantic Path Length). 



We derived 10 queries of increasing complexity in both structure 
and constraints (see brief description in Table 1, while complete 
queries can be be found at [12]). We broadcasted the above queries 
from one peer to the rest of the network (across all the acquain- 
tances), and measured the average and maximum time taken to 
translate each query along all semantic paths. Table 1 shows that 
the translation times stay within few seconds, and increasing query 
complexity (e.g. query Qio) slightly increases the translation time. 
In the second experiment, we aim at showing the impact of schemas 
heterogeneity on HepTox performance. Therefore, we vary the 
number of distinct heterogeneous schemas scattered around the net- 
work, and measure the average query translation time along all the 
semantic paths. As expected, Table 2 shows the query translation 
time goes from 2 (two heterogeneous schemas across the network), 
to 10 (ten heterogeneous schemas). Table 2 shows that the trans- 
lation time grows linearly with the number of schemas considered 
and that it also depends on the complexity of the translated query. 
For instance, queries Qs, Q~i, Q<s and Qio having double joins and 
multiple selections, have an higher translation time. The next ex- 
periment wants to show the minimal overhead introduced by our 
translation algorithm all along the query answering process. In Fig- 
ure 12(top), we measure the average time taken by each query to 
be completed. We highlight the various time components taken by 
query translation, network delay, and local query answering, re- 
spectively. It can be noted that query translation takes a negligible 
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Table 2: Query Translation times (ms) with varying number of 
distinct schemas (obviously equal to with 1 schema). 
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Figure 12: Query Answering Time Breakdown (top) and Time 
distribution of Answer Arrival at Source (bottom). 



time if compared to network delay and local query answering. Lo- 
cal query answering, essentially imputable to QIZX, was the bot- 
tleneck for all queries and caused the crashing of the most complex 
ones. Query answers to Qi, Qg and Qio show a cutoff point due 
to the fact that they never completed and were timed out. We tried 
different query engines before choosing QIZX, which is actually 
the fastest, thus this behavior was somehow outside our control. 

Query Performances in HepToX: To probe the impact of our 
query translation algorithm on the system's performance, we con- 
ducted an experiment in the Emulab environment in which a query 
is issued at an originating peer and spread across the entire net- 
work. We measured the percentage of query answers reaching the 
originating peer at each time interval, along with the percentage of 
peers completed. Figure 12(bottom) shows the results as a cumu- 
lative plot. The answers arrive in regular bursts just as expected in 
any normal P2P system. 

Finally, we conducted several experiments on simultaneous query 
processing, which is also feasible in HepTox. From these experi- 
ments, we realized that, similarly to what happens when one query 
is issued, the main component of time with multiple issued queries, 



is caused by local query answering, while the translation time still 
stays small. For lack of space, we omit these results. 

6. RELATED WORK 

P2P data integration systems We discuss here Piazza [3, 4], Hy- 
perion [2], Clio [20] and Lenzerini's et al. [5]. Piazza [3, 4] defines 
semantic LAV/GAV-style mappings among peer schemas. Query 
answering semantics is based on the notion of certain answers [3], 
and is based on query answering using views [15], a problem not 
yet completely solved for XML. Our mappings are easier and more 
intuitive to specify, and have a data exchange semantics (similar 
to [2]) rather than a view semantics. Query answering also follows 
the same principle. In Piazza, mappings need to be provided in an 
XQuery-like language, while in HepTox we can infer them from 
diagrams. Hyperion [2] combines mapping expressions and map- 
ping tables. Mapping tables being value-based correspondences 
across the world, are orthogonal to mapping expressions. Map- 
ping tables provide a lightweight mechanism for sharing data. Our 
Datalog-like mapping rules follow the same direction but capture 
schema correspondences rather than data value ones. Clio [20, 
25] also deals with XML data integration by considering seman- 
tically independent target and source nested schemas enriched with 
nested referential constraints. We found that this is very similar 
in spirit with what HepTox does. However, [20, 25, 3, 4] are 
limited in the heterogeneity they deal with and do not capture the 
class of XML transformations involving data schema interplay (see 
Figure 1). Lenzerini et al. [5] address the problem of data inter- 
operation in P2P systems using expressive schema mappings, also 
following the GAV/LAV paradigm, and show that the problem is in 
PTIME only when mapping rules are expressed in epistemic logic. 

Schema-matching systems Automated techniques for schema 
matching (e.g. CUPID [13], [9, 21]) are able to output elementary 
schema-level associations by exploiting linguistic features, context- 
dependent type matching, similarity functions etc. These associa- 
tions could constitute the input of our rule inference algorithm if 
the user does not provide the arrows. 

P2P systems with non-conventional lookups Popular P2P net- 
works, e.g. Kazaa, Gnutella, advertise simple lookup queries on file 
names. The idea of building full-fledged P2P DBMS is being con- 
sidered in many works. Internet-scale database queries and func- 
tionalities [16] as well as approximate range queries in P2P [11] 
and XPath queries in small communities of peers [10] have been 
extensively dealt with. All these works do not deal with reconcil- 
ing schema heterogeneity. [10] relies on a DHT-based network to 
address simple XPath queries, while [17] realizes IR-style queries 
in an efficient P2P relational database. 

7. CONCLUSIONS AND FUTURE WORK 

We have presented the HepToX P2P XML database system, focus- 
ing on the following key conceptual contributions: (i) an algorithm 
for inferring mapping rules from correspondences between hetero- 
geneous peer DTDs, specified via boxes and arrows; (ii) a precise 
and intuitive semantics for query evaluation in a P2P setting; (iii) a 
query translation algorithm that is correct w.r.t. this semantics and 
is efficient, as revealed by the detailed experimentation. We are 
currently investigating larger fragments of XQuery. A fundamental 
challenge is the reconstruction of the answers obtained from query 
translation in the schema of the originating peer. Another important 
milestone is handling 1-n and m-n correspondences between ele- 
ments across DTDs in the sense of [9]. A promising recent work in 



this direction is [7], It would be interesting to capture such complex 
mappings within our framework. 
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